#73 - Add DAQIRI + TensorRT example and benchmarks by RamyaGuru · Pull Request #213 · NVIDIA/daqiri

RamyaGuru · 2026-07-01T21:13:00Z

Closes #73

Adds an end-to-end example application under applications/resnet50_inference/ that wires DAQIRI's raw-Ethernet GPUDirect receive path to GPU inference: RX → GPU sequence-number reorder (image reassembly) → ResNet feature extraction via TensorRT (FP16) → per-class mean-feature stats, with no host copy on the RX→inference path. It runs over the DGX Spark p0→p1 cabled loopback and is platform-agnostic (also targets IGX / RTX Pro).

Builds only with -DDAQIRI_BUILD_APPLICATIONS=ON (off by default; requires TensorRT, i.e. the torch-base container), so it doesn't affect default builds.

What's included

Application (applications/resnet50_inference/): single-CUDA-stream RX loop, persistent per-image reorder buffer reusing DAQIRI's packet_reorder_copy_payload_by_sequence kernel via ReorderPipeline, double-buffered TensorRT runner, and a FeatureSink with two modes — example (real CIFAR-10 pcap → per-class mean-feature stats) and benchmark (synthetic frames → throughput sweep).
Tooling (tools/): ONNX export, CIFAR-10 pcap prep, and run_resnet_bench.sh sweeping ResNet-18/34/50/101/152 → resnet-bench.csv.
Per-batch latency instrumentation: CUDA timing events (batch-ready → features on host) reporting mean/p50/p99; the sweep also records RX-drop totals.
Tutorial (docs/tutorials/daqiri-resnet-inference.md): pipeline diagram + walkthrough (events, per-burst reorder flush, threads/cores map, shutdown drain), configuration callouts, and a DGX Spark results table.

Results (DGX Spark, FP16, batch 32, median of 3×30 s)

Model	img/s	Payload Gb/s	p50/p99 ms	RX drops
ResNet-18	2109	10.2	3.0 / 5.1	0
ResNet-50	1692	8.1	8.9 / 11.6	0
ResNet-152	1143	5.5	21.8 / 26.9	~19%

Throughput and latency track model depth; the pipeline is GPU-compute bound (top payload rate ~10 Gb/s vs the ~95 Gb/s the link sustains on the raw bench), and the RX-drop column marks the receive-bound → compute-bound transition — drop-free through ResNet-101, ~19% loss only at ResNet-152.

Testing

Verified on DGX Spark over the physical p0→p1 loopback (both example and benchmark modes). No unit tests (repo convention — verification via the benchmark executables).

Add applications/resnet50_inference/, an end-to-end example gated by the new DAQIRI_BUILD_APPLICATIONS CMake option (off by default; requires TensorRT): raw/DPDK GPUDirect RX -> GPU sequence-number reorder (image reassembly) -> ResNet feature extraction via TensorRT (FP16) -> per-class mean-feature stats. - Runs over the DGX Spark physical p0->p1 wire loopback (host_pinned MRs for GB10 unified memory; RX queue timeout_us flushes the dataset tail). - Example mode replays a CIFAR-10 pcap once and drains the full dataset (TensorRT engine is built/loaded before the run; RX worker exits on the expected image count and flushes the final partial inference batch). - Benchmark mode synthesizes frames for a throughput sweep across ResNet sizes. - Offline data prep + ONNX export + sweep tools under tools/. - Tutorial at docs/tutorials/daqiri-resnet-inference.md (added to nav; enables mermaid superfences). AGENTS.md documents the option and the binary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: rgurunathan <rgurunathan@nvidia.com>

Fill the benchmark-mode section with the DGX Spark p0->p1 sweep results (ResNet-18/34/50/101/152, FP16, batch 32, median of 3x30s reps): throughput and derived payload rate. Latency and RX-drop columns left as TBD pending instrumentation. Note the pipeline is GPU-compute bound across all models (payload rate well under link rate). Also rename the nav entry to 'DAQIRI + TensorRT Inference'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: rgurunathan <rgurunathan@nvidia.com>

greptile-apps · 2026-07-01T21:21:50Z

Greptile Summary

This PR adds an opt-in end-to-end example application (applications/resnet50_inference/) that wires DAQIRI's DPDK GPUDirect RX path directly into TensorRT ResNet inference, gated behind a new DAQIRI_BUILD_APPLICATIONS=OFF CMake option so it has no impact on default builds. The two bugs flagged in the previous review round (missing return after invalid-port detection in the RX worker, and TX burst not freed on a failed send_tx_burst) are addressed in the head commit.

Adds a single-CUDA-stream RX loop that reassembles images across burst boundaries via DAQIRI's sequence-number reorder kernel, batches them into an NCHW buffer, and runs TensorRT FP16 inference with double-buffered async D2H; burst ownership and the cudaStreamSynchronize-before-free invariant are correctly followed throughout.
Includes offline tooling (ONNX export, CIFAR-10 pcap prep, benchmark sweep script), two YAML configs, a tutorial with a Mermaid pipeline diagram and DGX Spark results table, and AGENTS.md / mkdocs.yml updates.

Confidence Score: 5/5

Safe to merge; the change is entirely additive (new opt-in application, off by default) and does not affect any existing build or runtime paths.

The application code is new, self-contained behind DAQIRI_BUILD_APPLICATIONS=OFF, and the two previously flagged correctness issues were fixed in the head commit. Burst ownership is correctly handled on all exit paths in both the RX and TX workers. The only open item is a doc-sync gap: docs/index.md and docs/landing/ were not updated to reflect the new mkdocs.yml nav entry.

mkdocs.yml — the new tutorial nav entry should be reflected in docs/index.md and docs/landing/ per the doc-sync rule. Also worth confirming whether docs/getting-started.md needs the new DAQIRI_BUILD_APPLICATIONS option added to its CMake-options table.

Important Files Changed

Filename	Overview
applications/resnet50_inference/inference_pipeline.cu	RX worker: reorder → NCHW batch → TRT inference loop. Missing-port-guard and burst-free on failed send fixed in head commit. Burst ownership (free after stream sync), cross-burst image accumulation, and partial-batch drain at shutdown all look correct.
applications/resnet50_inference/pcap_replayer.cpp	TX worker and pcap loader. All TX burst ownership paths (failed availability check, failed get, failed fill, failed send, successful send) correctly free the burst at the right lifecycle point.
applications/resnet50_inference/trt_runner.cu	TensorRT engine build/cache and double-buffered async D2H inference. Latency instrumented with CUDA timing events. Double-buffer accounting and drain_final shutdown path look correct.
applications/resnet50_inference/CMakeLists.txt	TensorRT discovery, yaml-cpp resolution, CUDA arch auto-detect, and target linkage. Gracefully skips the target (warning, not error) when TRT is absent, preserving the default build.
applications/resnet50_inference/main.cpp	Arg parsing, daqiri_init, RX/TX thread coordination. Waits on rx_ready before starting TX (defers timer past engine build), handles replay-once vs. loop modes cleanly.
mkdocs.yml	Adds tutorial nav entry and enables mermaid superfences. The matching docs/index.md and docs/landing/ landing-page links were not updated per the doc-sync rule.
docs/tutorials/daqiri-resnet-inference.md	Comprehensive tutorial covering the pipeline diagram, CUDA-event buffer design, per-burst reorder flush, threads/cores map, and shutdown drain. DGX Spark results table is well-annotated.
CMakeLists.txt	Adds DAQIRI_BUILD_APPLICATIONS option (OFF by default) and gates add_subdirectory(applications) behind it. Does not affect any existing build paths.

_{Reviews (3): Last reviewed commit: "#73 - Fix RX port guard and TX burst lea..." | Re-trigger Greptile}

Address PR #213 review feedback: - inference_rx_worker: return (and release the stream/events/batch buffer) when the RX interface can't be resolved, instead of falling through into the receive loop with port_id = -1. - pcap_tx_worker: free the burst on a failed send_tx_burst so a persistent send error doesn't drain the TX mempool one slot at a time. - tutorial: label the Gb/s column "Throughput (Gb/s)" to match the DGX Spark performance report's formatting. Assisted-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>

Instrument per-batch inference latency with CUDA timing events (batch-ready -> features on host, one-batch-late double-buffered) in the TRT runner, and print a mean/p50/p99 summary line. Extend run_resnet_bench.sh to record lat_p50_ms, lat_p99_ms and the cumulative RX-drop total per run. Expand the tutorial with the DGX Spark results table (throughput, payload rate, p50/p99 latency, RX drops) and its receive-bound -> compute-bound reading, a reorder config callout, a threads/cores map, the per-burst reorder flush, and the shutdown drain. Tighten the config YAML comments (drop the host_pinned block, shorten timeout_us). Assisted-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>

Address PR #213 review feedback: - inference_rx_worker: return (and release the stream/events/batch buffer) when the RX interface can't be resolved, instead of falling through into the receive loop with port_id = -1. - pcap_tx_worker: free the burst on a failed send_tx_burst so a persistent send error doesn't drain the TX mempool one slot at a time. - tutorial: label the throughput columns (img/s and Gb/s) to match the DGX Spark performance report, preserving unit case via an inline text-transform:none span (the theme uppercases table headers, which would otherwise render "Gb/s" as "GB/S"). Assisted-by: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>

RamyaGuru and others added 2 commits June 29, 2026 15:22

RamyaGuru marked this pull request as ready for review July 1, 2026 21:16

greptile-apps Bot reviewed Jul 1, 2026

View reviewed changes

Comment thread applications/resnet50_inference/inference_pipeline.cu

Comment thread applications/resnet50_inference/pcap_replayer.cpp

RamyaGuru added 2 commits July 1, 2026 17:46

RamyaGuru force-pushed the 73-resnet-inference-example branch from bb9a9db to a0f8861 Compare July 1, 2026 21:58

RamyaGuru marked this pull request as draft July 2, 2026 13:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

#73 - Add DAQIRI + TensorRT example and benchmarks#213

#73 - Add DAQIRI + TensorRT example and benchmarks#213
RamyaGuru wants to merge 4 commits into
mainfrom
73-resnet-inference-example

RamyaGuru commented Jul 1, 2026

Uh oh!

greptile-apps Bot commented Jul 1, 2026 •

edited

Loading

Greptile Summary

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

RamyaGuru commented Jul 1, 2026

What's included

Results (DGX Spark, FP16, batch 32, median of 3×30 s)

Testing

Uh oh!

greptile-apps Bot commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

greptile-apps Bot commented Jul 1, 2026 •

edited

Loading